25 05 2021

Helicopter view

  • project based on the article “Institutional Underpinnings and Democracy Backsliding in the Perspective of COVID-19” and the poster “A study of Democracy Backsliding in the Perspective of COVID-19 using econometrics and machine learning approaches” by PhD Jacek Lewkowicz, Michał Woźniak and Michał Wrzesiński
  • goal of the article: investigating the drivers of the impact of the political decisions, as a response to the COVID-19 pandemic
  • materials: use of a novel global dataset covering the period of the first wave of the pandemic (March-June 2020)
  • methods: various econometric and machine learning tools
  • conclusions: rule of law and high level of the initial state of democracy prevent from democracy backsliding during the pandemic

Goals of the reproducible project

  • reproduction of the initial results
  • improvement of the study by performing robustness checks
  • structuring the project
  • clean and readable code for reproducibility
  • combination of the research methodology developed as a part of a scientific article and a poster from conference MLinPL

Challenges

  • updated dependent variables:
    • new methodology of calculating indexes
    • wider time horizon
  • lack of reproducibility of the code (files sent by e-mails)
  • econometric part of the study (diagnostic tests) performed initially in STATA
  • mix of R and Python:
    • econometric part in R
    • creating databases and performing ML part in Python

Research purpose

  • COVID-19 and its economic and legal consequences are undoubtedly a very important problem that we have to face.
  • Governments are using the global pandemic to pursue their opportunistic goals, corruption and power strengthening.
    • Therefore, pandemic is concerned as a serious threat to democracy, as governments may try to limit democratic rules under the cover of pandemic management.
    • On the other hand, well-managed lockdowns and other means of government interventionism may lead to increased satisfaction with democracy or trust in government.
  • An alleged democracy recession can be observed in recent years and this backsliding is deeply rooted in institutional environment.
  • The phenomenon of democracy backsliding remains unexplained despite many attempts.

Research question




What is the relevance of the law and the current state of democracy for susceptibility to democracy backsliding in the face of the COVID-19?

Materials

Group of variables Variables Time range
Pandemic data Pandemic Democratic Violations Index (pandem), Pandemic Democratic Violations Disinformation Index (pandem_dis), Pandemic Backsliding Index (panback) 03.2020 - 07.2020
Democratization data Rule of law index (rule), Electoral democracy index (polyarchy), Education level index (education), Politico-geographic regions (region_geo) 2019
Economic & Demographic data Gini Index (gini), GDP pc (gdp_pc), Trade GDP (trade_gdp), Inflation rate (inflation), Oil sale (oil), Minerals sale (mineral),Population Density (density), Income group (income_group) mostly 2019
Fractionalization data Ethnic (ethnic_frac), Linguistic (ling_frac) and Religious fractionalization (relig_frac) 2002

Box plots

Adjustments

  • adding new variable (ratio of covid cases to the population)
  • adding squared terms for panback regression
  • robustness checks:
    • time periods (Q2, Q3 and Q4 of 2020)
    • collinearity (excluding variables that cause too much collinearity)
    • missing values (excluding variables with too much imputations)

OLS regression

  • specification with categorical variables
  • specification without categorical variables
  • robustness checks

Initial results - OLS with categorical variables

OLS models:
(1) (all variables)(2) (all variables, excluding polyarchy)(3) (all variables, excluding rule)
panbackpandempandem_dispanbackpandempandem_dispanbackpandempandem_dis
rule0.0670.0330.0270.119**-0.105*-0.097*---
polyarchy0.091-0.239**-0.216**---0.145**-0.212***-0.194***
education-0.008-0.0003-0.003-0.006-0.005-0.006-0.008-0.001-0.003
ethnic_frac0.057-0.00030.0130.0460.0290.0400.0660.0040.017
ling_frac0.0170.024-0.0060.0260.0003-0.0270.0150.022-0.007
relig_frac0.022-0.015-0.0060.019-0.0070.0020.024-0.013-0.005
gdp_pc_log0.0100.0190.0190.0040.0370.0350.0170.0220.022
gini_log0.0470.0750.0750.0410.0900.0890.0460.0740.075
inflation-0.00020.00000-0.0001-0.0001-0.0001-0.0002-0.0002-0.00003-0.0002
trade_gdp_log-0.045*-0.068**-0.068***-0.043*-0.072**-0.071***-0.044*-0.067**-0.067***
oil0.0020.0010.0010.0020.0010.0010.0020.0010.001
mineral0.0040.006**0.006**0.0040.006*0.006*0.0030.006**0.006**
density_log0.0130.0100.0100.0140.0090.0090.0130.0100.010
region_geo FEyesyesyesyesyesyesyesyesyes
income_group FEyesyesyesyesyesyesyesyesyes
Constant-0.1360.0680.047-0.066-0.114-0.118-0.1700.0510.033
Observations146146146146146146146146146
Adjusted R20.2180.3180.3130.2160.2830.2810.2180.3230.318
Note:*p<0.1; **p<0.05; ***p<0.01

Initial results - OLS without categorical variables

OLS models:
(1) (all variables)(2) (all variables, excluding polyarchy)(3) (all variables, excluding rule)
panbackpandempandem_dispanbackpandempandem_dispanbackpandempandem_dis
rule0.0190.020-0.0080.053-0.148***-0.152***---
polyarchy0.055-0.272***-0.233***---0.069-0.257***-0.239***
education-0.0030.0010.0005-0.0030.0020.001-0.0030.0010.001
ethnic_frac0.029-0.0130.0090.028-0.0080.0140.030-0.0120.009
ling_frac0.0340.0350.0080.0320.0440.0150.0350.0360.007
relig_frac-0.029-0.044-0.030-0.026-0.060-0.044-0.029-0.044-0.030
gdp_pc_log-0.014-0.002-0.001-0.014-0.005-0.003-0.013-0.0004-0.001
gini_log0.0560.099*0.094*0.0540.109*0.102*0.0560.099*0.094*
inflation-0.0003-0.0001-0.0003-0.0003-0.0003-0.0004-0.0003-0.0001-0.0003
trade_gdp_log-0.019-0.039*-0.039*-0.019-0.037-0.038*-0.019-0.039*-0.039*
oil0.001-0.0001-0.00020.00020.0010.0010.0005-0.0002-0.0002
mineral0.0010.0030.0030.0010.0030.0020.0010.0030.003
density_log0.015*0.018**0.014*0.014*0.023**0.018**0.015*0.018**0.013*
region_geo FEnonononononononono
income_group FEnonononononononono
Constant0.0190.0550.0650.037-0.035-0.0120.0080.0430.070
Observations146146146146146146146146146
Adjusted R20.0570.2760.2760.0590.2020.2150.0630.2810.282
Note:*p<0.1; **p<0.05; ***p<0.01

Reproduced results - OLS with categorical variables

OLS models:
(1) (all variables)(2) (all variables, excluding polyarchy)(3) (all variables, excluding rule)
panbackpandempanbackpandempanbackpandem
rule0.539***-0.0210.734***-0.149**--
rule2-0.506***--0.676***---
polyarchy0.702***-0.216**--0.977***-0.233***
polyarchy2-0.696***----0.965***-
cases_ratio0.00000-0.000000.00000-0.00000-0.00000-0.00000
education_log-0.020-0.026-0.026-0.047-0.034-0.024
ethnic_frac-0.051-0.001-0.0590.0320.009-0.004
ling_frac0.0270.0190.044-0.0050.0290.021
relig_frac-0.006-0.0300.008-0.0210.001-0.031
gdp_pc_log0.0240.0150.0160.0310.0280.013
gini_log0.0670.1410.0600.165*0.0910.140
inflation0.0005*0.001*0.001**0.0010.00010.001*
trade_gdp_log0.014-0.0030.026-0.0050.005-0.003
oil-0.001-0.004*-0.001-0.003-0.001-0.004*
mineral-0.00030.002-0.00010.002-0.00040.002
density_log0.0050.0080.0110.0070.0030.008
region_geo FEyesyesyesyesyesyes
income_group FEyesyesyesyesyesyes
Constant-0.650*-0.366-0.563-0.608-0.693*-0.349
Observations143143143143143143
Adjusted R20.3350.2750.2860.2520.2810.281
Note:*p<0.1; **p<0.05; ***p<0.01

Reproduced results - OLS without categorical variables

OLS models:
(1) (all variables)(2) (all variables, excluding polyarchy)(3) (all variables, excluding rule)
panbackpandempanbackpandempanbackpandem
rule0.475***-0.0640.734***-0.199***--
rule2-0.459***--0.708***---
polyarchy0.721***-0.221***--1.081***-0.268***
polyarchy2-0.726***----1.121***-
cases_ratio0.000000.000000.000000.00000-0.000000.00000
education_log-0.0090.001-0.016-0.0003-0.0040.006
ethnic_frac-0.0410.013-0.0430.023-0.0070.010
ling_frac0.0510.0420.0600.0460.0440.039
relig_frac-0.054*-0.099*-0.035-0.110**-0.073*-0.100*
gdp_pc_log0.0100.00050.0020.00050.012-0.005
gini_log0.079**0.113*0.086*0.116*0.082*0.113*
inflation0.0004*0.00050.00050.00030.000030.001
trade_gdp_log-0.004-0.0040.007-0.003-0.011-0.005
oil-0.001**-0.003-0.002-0.001-0.001-0.003
mineral-0.001-0.001-0.002-0.001-0.001-0.001
density_log0.0100.0070.0110.0100.0100.006
region_geo FEnononononono
income_group FEnononononono
Constant-0.432***-0.016-0.355*-0.084-0.422**0.019
Observations143143143143143143
Adjusted R20.3360.2520.2780.2110.2850.253
Note:*p<0.1; **p<0.05; ***p<0.01

Robustness check - collinearity




OLS models:
(1)(2)(3)
panbackpandempanbackpandempanbackpandem
rule0.538***-0.0420.769***-0.175***--
rule2-0.510***--0.724***---
polyarchy0.728***-0.220**--1.050***-0.256***
polyarchy2-0.723***----1.059***-
Control variables:cases_ratio, education_log, fractionalization, gdp_pc_log, inflation, trade_gdp_log, oil, mineral, density_log, region_geo
Note:*p<0.1; **p<0.05; ***p<0.01

Robustness check - collinearity




OLS models:
(1)(2)(3)
panbackpandempanbackpandempanbackpandem
rule0.491***-0.0600.744***-0.206***--
rule2-0.474***--0.719***---
polyarchy0.726***-0.237***--1.108***-0.281***
polyarchy2-0.737***----1.158***-
Control variables:cases_ratio, education_log, fractionalization, gdp_pc_log, inflation, trade_gdp_log, oil, mineral, density_log
Note:*p<0.1; **p<0.05; ***p<0.01

Robustness check - missings




OLS models:
(1)(2)(3)
panbackpandempanbackpandempanbackpandem
rule0.575***-0.0060.773***-0.158**--
rule2-0.535***--0.719***---
polyarchy0.702***-0.254***--1.011***-0.259***
polyarchy2-0.717***----1.025***-
Control variables:cases_ratio, ethnic_frac, ling_frac, relig_frac, gdp_pc_log, inflation, trade_gdp_log, oil, mineral, density_log, region_geo, income_categories
Note:*p<0.1; **p<0.05; ***p<0.01

Robustness check - missings




OLS models:
(1)(2)(3)
panbackpandempanbackpandempanbackpandem
rule0.478***-0.0610.751***-0.198***--
rule2-0.457***--0.719***---
polyarchy0.752***-0.224***--1.110***-0.270***
polyarchy2-0.762***----1.151***-
Control variables:cases_ratio, ethnic_frac, ling_frac, relig_frac, gdp_pc_log, inflation, trade_gdp_log, oil, mineral, density_log
Note:*p<0.1; **p<0.05; ***p<0.01

Random Forest

  • SHAP summary plots
  • Permutation importance
  • Partial Dependence Plots (PDPs) (1 dimensional and 2 dimensional)
  • Accumulated Local Effects (ALE) plots (1 dimensional and 2 dimensional)

SHAP summary plots - panback

SHAP summary plots - pandem

Permutation Importance - panback

Permutation Importance - pandem

1D PDP - panback vs polyarchy

1D PDP - pandem vs polyarchy

2D PDP - panback vs polyarchy and rule

2D PDP - pandem vs polyarchy and rule

CatBoost

  • SHAP summary plots
  • Permutation importance
  • Partial Dependence Plots (PDPs) (1 dimensional and 2 dimensional)
  • Accumulated Local Effects (ALE) plots (1 dimensional and 2 dimensional)

SHAP summary plots - panback

SHAP summary plots - pandem

Permutation Importance - panback

Permutation Importance - pandem

1D PDP - panback vs polyarchy

1D PDP - pandem vs polyarchy

2D PDP - panback vs polyarchy and rule

2D PDP - pandem vs polyarchy and rule

Conclusions

  • We recreated the original results and updated them with the new data and the new methodology.

  • We confirmed that the results are not distorted by several robustness checks.

  • Results compared to the original study are analogous.

  • Influence of law and state of democracy is exceptionally strong. The stronger the rule of law and the higher the level of democracy, the lower is the risk of democracy backsliding in the face of global pandemic.

  • Our study provides other variables that are worth further investigation such as: GDP per capita, population density, trade as a share of GDP, economic inequality, mineral rents as a share of GDP and linguistic fractionalization.

Backup - RF

1D ALE - panback vs polyarchy

1D ALE - pandem vs polyarchy

2D ALE - panback vs polyarchy and rule

2D ALE - pandem vs polyarchy and rule

Backup - CatBoost

1D ALE - panback vs polyarchy

1D ALE - pandem vs polyarchy

2D ALE - panback vs polyarchy and rule

2D ALE - pandem vs polyarchy and rule